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hierarchy of items is stored in a search priority order. 
Multiple element definitions and groups of elements are 
identified. Representations of the element definitions and 
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Protocol security associations. The searchable data structure 
may include an associative memory or a plurality of asso- 
ciative memory entries. 
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STRONG AND SEARCHING A HIERARCHY hosts or between two security gateways, two security asso- 

OF ITEMS OF PARTICULAR USE WITH IP ciations (one in each direction) are required. A security 

SECURITY POLICIES AND SECURITY association is uniquely identified by a triple consisting of a 

ASSOCIATIONS Security Parameter Index (SPI), an IP Destination Address, 

5 and a security protocol identifier. In principle, the destina- 
tion address may be a unicast address, an IP broadcast 

TECHNICAL FIELD address, or a multicast group address. The set of security 

One embodiment of the invention especially relates to services offered b ? an S ^ de Pf nd ? on * e L se f u A rit y P 10 **? 1 
communications and computer systems; and more selected, the S A mode, the endpoints of the S A, and on the 
particularly, one embodiment relates to storing and search- 10 election of °P tlonal service f ™ ih ] n * he P rotocoL 
ing a hierarchy of items which may be particularly useful for example, one security protocol provides data origin authen- 
implementing security policies and security associations, tication ™ d connectionless mtegnty for IP datagrams, 
such as, but not limited to Internet Protocol security (IPsec) The IP datagrams transmitted over an individual SA are 
in routers, packet switching systems, computers, and/or afforded protection by exactly one security protocol. Some- 
other devices. 15 times a security policy may call for a combination of 

services for a particular traffic flow that is not achievable 

BACKGROUND with a single SA. In such instances it will be necessary to 

The communications industry is rapidly changing to employ multiple SAs to implement the required security 

adjust to emerging technologies and ever increasing cus- P olicv - ^ term "security association bundle" or "SA 

tomer demand. This customer demand for new applications 20 bundle" is applied to a sequence of SAs through which 

and increased performance of existing applications is driv- traffic must be processed to satisfy a security policy. The 

ing communications network and system providers to order of the sequence is defined by the pohcy. (Note that the 

employ networks and systems having greater speed and SAs that comprise a bundle may terminate at different 

capacity (e.g., greater bandwidth). In trying to achieve these endpoints. For example, one SA may extend between a 

goals, a common approach taken by many communications 25 moblle host and a security gateway and a second, nested SA 

providers is to use packet switching technology. may extend to a host behmd the gateway.) 

Increasingly, public and private communications networks RFC 2401 defines that there are two nominal databases in 

are being built and expanded using various packet the IPsec general model, with these two databases being the 

^s-k technologies, such as Internet P rotocol HP\ security policy database (SPD) and the security association 

(OJ . Asecurity architecture for th( 7nternet. ProtocpT flPsec) is 30 database (SAD). The former specifies the policies that 

J^MpfAlX defined in. S. KENT and R. ATKINSON, u t>ecunty Archi- determine the disposition of all IP traffic inbound or out- 

TtfT>7 I lecture for IP," RFC 2401, November 1998, which is hereby hound from a host, security gateway, or BITS or BITW 

nrmSLQ I incorporated by reference, Ip sec implementation. The latter database contains param- 

H U .V An IPsec implementation operates in a host or a security 35 eterc * at ™ f 500 ^ f ih '* ch < activ '> purity associa- 

/M 0**^ § atewa y environment, affording protection to IP traffic. The *«^h» ^ define * J" -concept of a elector, a set 

t X*VAi P rotectlon offered * based on requirements defined by a of IP and u PP er P rotoco1 fi f d ™ lues tha . ls «?*by Je 

tyW^ J Security Policy Database (SPD) established and maintained security policy database to map traffic to a policy, i.e., an SA 

by a user or system administrator, or by an application ( or SA Dundle )- 

operating within constraints established by either of the 40 Each interface for which IPsec is enabled requires nomi- 

above. In general, packets are selected for one of three nally separate inbound vs. outbound databases (SAD and 

processing modes based on IP and transport layer header SPD), because of the directionality of many of the fields that 

information matched against entries in the database. Each are used as selectors. Typically there is just one such 

packet is either afforded IPsec security services, discarded, interface, for a host or security gateway (SG). Note that an 

or allowed to bypass IPsec, based on the applicable database 45 SG would always have at least two interfaces, but the 

policies. "internal" one to the corporate net, usually would not have 

IPsec provides security services at the IP layer by IPsec enabled and so only one pair of SADs and one pair of 

enabling a system to select required security protocols, SPDs would ** needed - 0n the ot 1 her 1 hand ' lfa hos J nad 

determine the algorithm(s) to use for the service(s), and put m ultiple interfaces or an SG had multiple external interfaces, 

in place any cryptographic keys required to provide the 50 11 mi & ht be necessary to have separate SAD and SPD pairs 

requested services. IPsec can be used to protect one or more * or each interface. 

"paths" between a pair of hosts, between a pair of security Ultimately, a security association is a management con- 
gateways, or between a security gateway and a host. The set struct used to enforce a security policy in the IPsec envi- 
of security services that IPsec can provide includes access ronment. Thus, an essential element of SA processing is an 
control, connectionless integrity, data origin authentication, 55 underlying Security Policy Database (SPD) that specifies 
rejection of replayed packets (a form of partial sequence what services are to be offered to IP datagrams and in what 
integrity), confidentiality (encryption), and limited traffic fashion. The form of the database and its interface are 
flow confidentiality. Because these services are provided at outside the scope of RFC 2401. However, RFC 2401 does 
the IP layer, they can be used by any higher layer protocol, specify certain minimum management functionality that 
e.g., TCP, UDP, ICMP, BGP, etc. 60 m ust be provided, to allow a user or system administrator to 
IPsec packet' classification is specified as a two-layer control how IPsec is applied to traffic transmitted or received 
hierarchy: the relevant security policy (SP) must be found by a host or transiting a security gateway, 
first out of an ordered list of SPs, and then within the context The SPD must be consulted during the processing of all 
of the located SP, the correct security association (SA) must traffic (inbound and outbound), including non-IPsec traffic. 
/r -^ be found. A security association is a simplex "connection" 65 In order to support this, the SPD requires distinct entries for 
f2>) that affords s ecurity services to the traffic carried by it. To inbound and outbound traffic. The SPD contains an ordered 
^—s secure typical ^bidirectionaj^ communication between two list of policy entries. Each policy entry is keyed by one or 
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more selectors that define the set of IP traffic encompassed may be particularly useful for implementing security poli- 
by this policy entry. One can think of this as separate SPDs cies and security associations, such as, but not limited to 
(inbound vs. outbound). In addition, a nominally separate Internet Protocol security (IPsec) in routers, packet switch- 
SPD must be provided for each IPsec-enabled interface. A ing systems, computers, and/or other devices. 
SPD must discriminate among traffic that is afforded IPsec 5 One embodiment stores a hierarchy of items in a search 
protection and traffic that is allowed to bypass IPsec. This priority order. Multiple element definitions and groups of 
applies to the IPsec protection to be applied by a sender and elements are identified. Representations of the element defi- 
to the IPsec protection that must be present at the receiver. nitions and elements are stored in a prioritized searchable 
For any outbound or inbound datagram, three processing data structure in decreasing search priority such that repre- 
choices are possible: discard, bypass IPsec, or apply IPsec. 10 sentations of each particular element definition is stored 
The first choice refers to traffic that is not allowed to exit the after representations of a set of particular elements associ- 
host, traverse the security gateway, or be delivered to an a ted with the particular element definition and before rep- 
application at all. The second choice refers to traffic that is resentations of lower priority element definitions and their 
allowed to pass without additional IPsec protection. The associated elements. In one embodiment, the element defi- 
third choice refers to traffic that is afforded IPsec protection, 15 nitions include Internet Protocol security policies and the 
and for such traffic the SPD must specify the security elements include Internet Protocol security associations. In 
services to be provided, protocols to be employed, algo- one embodiment, the searchable data structure includes an 
rithms to be used, etc. associative memory or a plurality of associative memory 

In each IPsec implementation there is a nominal security entries. In one embodiment, an element definition or element 
association database, in which each entry defines the param- 20 corresponding to a range of values is split into multiple 
eters associated with one SA. Each SA has an entry in the entries. In one embodiment, the hierarchy includes more 
SAD. For outbound processing, entries are pointed to by than two levels, and the element definitions and groups of 
entries in the SPD. Note that if an SPD entry does not elements are just two of the more than two levels, 
currently point to an SAthat is appropriate for the packet, the One embodiment maintains a data structure for an iden- 
implementation creates an appropriate SA (or SA Bundle) 25 tified ordered list of Internet Protocol security policies, 
and links the SPD entry to the SAD entry. For inbound Ordered associative memory entries associated with the 
processing, each entry in the SAD is indexed by a destina- ordered list of Internet Protocol security policies are pro- 
tion IP address, IPsec protocol type, and SPI. The following grammed into one or more associative memories. Corre- 
parameters are associated with each entry in the SAD. This sponding context memory entries associated with the 
description does not purport to be a MIB, but only a 30 ordered list of Internet Protocol security policies are pro- 
specification of the minimal data items required to support grammed into one or more context memories. An associative 
an SA in an IPsec implementation. memory lookup operation is performed on the ordered 

FIG. 1 illustrates a prior art implementation based on RFC associative memory entries based on a received packet to 

2401 for processing an outbound packet. Processing begins identify a particular associative memory entry location. A 

with process block 100, and proceeds to process block 102, 35 lookup operation is performed on the context memory based 

wherein a database lookup operation is performed in the on the particular associative memory entry location to 

security policy database based on the packet to identify the identify a particular Internet Protocol security policy of the 

corresponding security policy. If no policy is found as ordered list of Internet Protocol security policies. A particu- 

determined in process block 104, then the packet is dropped lar security association entry based on the received packet is 

in process block 106, and processing is complete as indi- 40 added to the ordered associative memory entries, the par- /7Y\ 

: vD 



cated by process block 108. Otherwise, in process block 110, ticular secui&yassociation entry corresponding to the par- 

a second lookup operation is performed based on the packet, tic^lai furternetj Protocol security policy, and the particular l 

this time in the security association database corresponding security association entry being added to the ordered asso- ~Xjfytff\&**' 

to the security policy identified in the previous lookup ciative memory entries prior to the particular associative 

operation. As determined in process block 112, if a corre- 45 memory entry location and after other security policy entries 

sponding security association is not located, then in process of the ordered list of Internet Protocol security policies 

block 114, the security association is added to the corre- located prior to the particular associative memory entry 

sponding security association database. In process block location. 

116, the packet is processed according to the corresponding BRIEF DESCRIPTION OF THE DRAWINGS 
secunty association. Processing is complete as indicated by 

process block 118. The appended claims set forth the features of the inven- 

RFC 2401 defines a two-step process for performing tion ™ ih particularity. The invention, together with its 

lookup operations to in order to identify a SA associated advantages, may be best understood from the following 

with a packet, i.e., by first performing a lookup in a security „ detailed description taken in conjunction with the accom- 

policy database and then, performing a subsequent second panying drawings of which: 

lookup operation based on the identified security policy to FIG. 1 illustrates a prior art implementation of IPsec; 

identify the corresponding security association). Especially FIG. 2A is a block diagram illustrating one embodiment 

as packet rates and then number of packets to be processed for storing and searching a hierarchy of items; 

by a packet processor increases, this two-stage lookup 60 FIG. 2B is a block diagram illustrating one embodiment 

process can be limiting. Desired is a new way of performing for storing and searching a hierarchy of items; 

IPsec identification operations. FIG. 3A is a block diagram illustrating a prioritized 

searchable data structure used in one embodiment; 

SUMMARY FIG. 3B is a block diagram illustrating a prioritized 

Disclosed are, inter alia, methods, apparatus, data 65 searchable data structure used in one embodiment; 



structures, computer- readable medium, mechanisms, and FIG. 3C is a block diagram illustrating a prioritized 
means for storing and searching a hierarchy of items which searchable data structure used in one embodiment; 
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one embodiment, two security association databases are 
used to enhance performance. Outbound security processor 
442 processes each outbound packet by first extracting the 
five selectors specified in RFC 2401, and then performing a 
search for a match in TCAM 424. If a match is found, 
outbound security processor 442 indexes the context array 
using the index of the matched TCAM entry included in 
lookup results 433. The context array entry indicates 
whether the TCAM match corresponds to a matching S A or 
SP. If it is a SP, the context array also consists of the 
appropriate action for packet matching that SA. If it is a SA, 
the context array contains the index into the SAD for the 
corresponding SA. There is only one data structure of 
outbound SA. 

FIG. 5A illustrates associative memory entries used in one 
embodiment. As shown, TCAM entry 500 includes a source 
address field 501, a destination address field 502, a source 
port field 503, a destination port field 504, a protocol type 
field 505, a service indication field 506, an entry type field 
507 to indicate whether the entry is a SA or SP entry, and an 
implementation specific field 508. Note, one embodiment 
sets the mask field to don't care in field 507 if the entry 
corresponds to a service policy because every search is 
performed on the SPD (e.g., on all SP entries). By not 
masking out the value when the entry corresponds to an SA, 
then either all entries can be searched or only SPs can be 
searched. Thus, global mask register-0 510 has bits set to 
match in fields 511-516 and to ignore (i.e., don't card) in 
fields 517-518. Thus, using global mask register-0 510 in a 
search will cause both SP and SA entries to be searched. 
Global mask register-1 520 has bits set to match in fields 
521-527 and to ignore (i.e., don't card) in field 528. Thus, 
using global mask register-1 520 in a search with the lookup 
word specifying SP entry types, a search will cause only SP 
entries to be searched. Note, the use of block masks are 
described in Ross et al., "Block Mask Ternary CAM," U.S. 
Pat. No. 6,389,506, issued May 14, 2002, which is hereby 
incorporated by reference. 

FIG. 5B illustrates a process used in one embodiment for 
generating multiple associative memory entries for a corre- 
sponding range of values. Some applications desire to match 
on a range of values (e.g,., source port number 72-83). 

Because TCAMs do not support arbitrary sets or ranges as 
selection criteria, the splitter is required to perform any 
required entry expansion. For example, implementing the 
destination port ranges <25 and >25 requires splitting a 
single entry into sixteen entries. FIG. 5B illustrates pseudo 
code of a mechanism used in one embodiment to split entries 
into multiple entries. The splitter converts a SP specified in 
a range-set format into a SP specified in an expanded form 
using a collection of matching values and don't-care mask. 
For example, support a range of 1 to 15 becomes 4 sets of 
,> (matching values, don't care mask): ( 0x1. 0xe\ (0x2.0 xd > ). 
(0x4, Oxb), and (0x8, 0x7). As shown( Jrst TCAM^ ntry 
d . . . d is checked to see if it matches a subset or tne values 
a/\ covered by the range. If not, then the process is repeated 
Y ' 1 with Od . . . d and Id ... d. This happens recursively (using 
the stacks — not function recursion). Branches are trimmed 
when the entry being tested matches a disjoint set of values. 
Entries are saved when they match a subset of the values 
matched by the range. Entries that match overlapping sets 
are split and pushed onto the work stack. 

FIG. 6A illustrates a process used in one embodiment for 
processing an inbound packet. Processing begins with pro- 
cess block 600, and proceeds to process block 602, wherein 
a packet is received. As determined in process block 604, if 
the packet is marked as conforming to IPsec, then in process 
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block 606 the packet is processed, and processing is com- 
pleted as indicated by process block 619. Otherwise, in 
process block 610, a lookup word is generated based on the 
received packet (e.g., with fields in accordance to those 

5 stored in the associative memory or other implementations 
of the data structure). In process block 612, a lookup 
operation is initiated and performed in the associative 
memory using the lookup word and a global mask register 
such that only SP entries are searched. The lookup result is 

10 received and a lookup operation based on the result is 
performed in the context memory in process block 614. 
Then, in process block 616, the packet is processed accord- 
ing to the action identified in the context memory. Process- 
ing is complete as indicated by process block 619. 

15 FIG. 6B illustrates a process used in one embodiment for 
processing an outbound packet. Processing begins with 
process block 640, and proceeds to process block 642, 
wherein a packet is received. Next, in process block 644, a 
lookup word is generated based on the received packet.). In 

20 process block 646, a lookup operation is initiated and 
performed in the associative memory using the lookup word 
and a global mask register such that both SP and SA entries 
are searched. The lookup result is received and a lookup 
operation based on the result is performed in the context 

25 memory in process block 648. As determined in process 
block 650, if the entry matched corresponds to an SA entry, 
then in process block 652, the action to perform is identified 
in the SAD based on the lookup result retrieved from the 
context memory, and the packet is processed according to 

30 the identified action. Otherwise, in process block 660, the 
packet is processed according to the action identified by the 
context memory; and in process block 662, a security access 
entry is added to the SAD and the associative and context 
memories are updated accordingly. Processing is complete 

35 as indicated by process block 669. 

FIG. 7 illustrates a process used in one embodiment for 
adding an entry to an ordered list of associative memory 
entries. Processing begins with process block 700, and 
proceeds to process block 702, wherein an associative 

40 memory or other prioritized searchable data structure update 
request is identified. Next, in process block 704, the partition 
and possibly the exact location(s) to add one or more entries 
entry are identified. As determined in process block 706, if 
there is space to add the one or more entries in the identified 

45 partition, then the entries are added in process block 712. 
Otherwise, space for the new entries is made (or attempted 
to be made) in process block 708. As determined in process 
block 710, if this expansion of the partition was successful, 
then the then the entries are added in process block 712. 

50 Otherwise, there is no room for the entries and an error 
condition is generated. Processing is complete as indicated 
by process block 714. 

FIGS. 8A-D and 9A-D illustrate processes used in one 
embodiment for expanding partitions and redistributing 

55 space allocated to partitions. Note, these processes may call 
each in a recursive or other fashion to expand/shrink parti- 
tions to redistribute the free space among partitions. One 
embodiment attempts to maintain an even distribution of 
free space (or something approximating such) across all 

60 partitions to minimize the amount of adjusting to be per- 
formed in adding one or more entries to a partition. By 
maintaining an approximate even distribution of free space 
among partitions, a single insert of an element or element 
definition (which may include one or more associative 

65 memory entries) can be quickly performed and limits the 
worst-case insertion time, which is important for applica- 
tions with high update rates. Note, one embodiment does not 
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PSTORIiyq kND SEARCHING a hierarchy of items of particular 

USE WITH IP SECURITY POLICIES AND SECURITY ASSOCIATIONS 

TECHNICAL FIELD 

5 One embodiment of the invention especially relates to communications and 

computer systems; and more particularly, one embodiment relates to storing and 
searching a hierarchy of items which may be particularly useful for implementing security 
policies and security associations, such as, but not limited to Internet Protocol security 
(IPsec) in routers, packet switching systems, computers, and/or other devices. 

10 

BACKGROUND 

The communications industry is rapidly changing to adjust to emerging 
technologies and ever increasing customer demand. This customer demand for new 
applications and increased performance of existing applications is driving 

15 communications network and system providers to employ networks and systems having 
greater speed and capacity (e.g., greater bandwidth). In trying to achieve these goals, a 
common approach taken by many communications providers is to use packet switching 
technology. Increasingly, public and private communications networks are being built and 
expanded using various packet technologies, such as Internet Protocol (IP). 

20 A security architecture for th ^Internet Protocol jjlPsec) is defined in. S. KENT 

and R. ATKINSON, "Security Architecture for IP," RFC 2401, November 1998, which is 
hereby incorporated by reference. 

An IPsec implementation operates in a host or a security gateway environment, 
affording protection to IP traffic. The protection offered is based on requirements defined 

25 by a Security Policy Database (SPD) established and maintained by a user or system 
administrator, or by an application operating within constraints established by either of 
the above. In general, packets are selected for one of three processing modes based on IP 
and transport layer header information matched against entries in the database. Each 

1 
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packet is either afforded IPsec security services, discarded, or allowed to bypass IPsec, 
based on the applicable database policies. 

IPsec provides security services at the IP layer by enabling a system to select 
required security protocols, determine the algorithm(s) to use for the service(s), and put in 
5 place any cryptographic keys required to provide the requested services. IPsec can be used 
to protect one or more "paths" between a pair of hosts, between a pair of security 
gateways, or between a security gateway and a host. The set of security services that IPsec 
can provide includes access control, connectionless integrity, data origin authentication, 
rejection of replayed packets (a form of partial sequence integrity), confidentiality 
10 (encryption), and limited traffic flow confidentiality. Because these services are provided 
at the IP layer, they can be used by any higher layer protocol, e.g., TCP, UDP, ICMP, 
BGP, etc. 

IPsec packet classification is specified as a two-layer hierarchy: the relevant 
security policy (SP) must be found first out of an ordered list of SPs, and then within the 

15 context of the located SP, the correct security association (SA) must be found. A security 
association is a simplex "connection" that affords security services to the traffic carried by 
it. To secure typical ^bi-directionay communication between two hosts or between two 
security gateways, two security associations (one in each direction) are required. A 
security association is uniquely identified by a triple consisting of a Security Parameter 

20 Index (SPI), an IP Destination Address, and a security protocol identifier. In principle, the 
destination address may be a unicast address, an IP broadcast address, or a multicast 
group address. The set of security services offered by an SA depends on the security 
protocol selected, the S A mode, the endpoints of the S A, and on the election of optional 
services within the protocol. For example, one security protocol provides data origin 

25 authentication and connectionless integrity for IP datagrams. 

The IP datagrams transmitted over an individual SA are afforded protection by 
exactly one security protocol. Sometimes a security policy may call for a combination of 
services for a particular traffic flow that is not achievable with a single SA. In such 
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security policy of the ordered list of Internet Protocol security policies. A particular 
security association entry based on the received packet is added to the ordered associative 
memory entries, the particular security association entry corresponding to the particular 
[ InternetJ Protocol security policy, and the particular security association entry being added 
5 to the ordered associative memory entries prior to the particular associative memory entry 
location and after other security policy entries of the ordered list of Internet Protocol 
security policies located prior to the particular associative memory entry location. 
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don't care mask): (0x1, Oxe), (0x2,0xd), (0x4, Oxb), and (0x8, 0x7). As showif first, the 



TCAMJentry d...d is checked to see if it matches a subset of the values covered by the 
range. If not, then the process is repeated with 0d...d and ld...d. This happens recursively 
(using the stacks - not function recursion). Branches are trimmed when the entry being 
5 tested matches a disjoint set of values. Entries are saved when they match a subset of the 
values matched by the range. Entries that match overlapping sets are split and pushed 
onto the work stack. 

FIG. 6A illustrates a process used in one embodiment for processing an inbound 
packet. Processing begins with process block 600, and proceeds to process block 602, 

10 wherein a packet is received. As determined in process block 604, if the packet is marked 
as conforming to IPsec, then in process block 606 the packet is processed, and processing 
is completed as indicated by process block 619. Otherwise, in process block 610, a 
lookup word is generated based on the received packet (e.g., with fields in accordance to 
those stored in the associative memory or other implementations of the data structure). In 

15 process block 612, a lookup operation is initiated and performed in the associative 

memory using the lookup word and a global mask register such that only SP entries are 
searched. The lookup result is received and a lookup operation based on the result is 
performed in the context memory in process block 614. Then, in process block 616, the 
packet is processed according to the action identified in the context memory. Processing 

20 is complete as indicated by process block 619. 

FIG. 6B illustrates a process used in one embodiment for processing an outbound 
packet. Processing begins with process block 640, and proceeds to process block 642, 
wherein a packet is received. Next, in process block 644, a lookup word is generated 
based on the received packet. ). In process block 646, a lookup operation is initiated and 

25 performed in the associative memory using the lookup word and a global mask register 
such that both SP and S A entries are searched. The lookup result is received and a lookup 
operation based on the result is performed in the context memory in process block 648. 
As determined in process block 650, if the entry matched corresponds to an SA entry, 
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